Conditional provisioning of auxiliary information with a media presentation

ABSTRACT

Techniques for conditional presentation of auxiliary information are provided. In one example, a computer-implemented method comprises determining, by a device operatively coupled to a processor, feedback information for a user in response to a media presentation, wherein the feedback information is indicative of a mental state of the user during presentation of the media presentation. The computer-implemented method can further comprise determining content of the media presentation associated with the mental state of the user, and determining auxiliary information about the content based on the mental state of the user. The computer-implemented method can further comprise providing, by the device, the content to a user device during the presentation of the media presentation and for rendering at the user device in association with presentation of the content.

BACKGROUND

The subject disclosure relates to conditional provisioning of auxiliaryinformation with a media presentation.

SUMMARY

The following presents a summary to provide a basic understanding of oneor more embodiments of the invention. This summary is not intended toidentify key or critical elements, or delineate any scope of theparticular embodiments or any scope of the claims. Its sole purpose isto present concepts in a simplified form as a prelude to the moredetailed description that is presented later. In one or moreembodiments, systems, computer-implemented methods, apparatus and/orcomputer program products that facilitate conditional provisioning ofauxiliary information with a media presentation are described.

According to an embodiment, a system is provided that can comprise amemory that stores computer-executable components and a processor thatexecutes computer-executable components stored in the memory. In one ormore implementations, the computer-executable components comprise afeedback component that receives feedback information indicative of amental state of a user during presentation of a media presentation. Thecomputer-executable components can further comprise an analysiscomponent that determines content of the media presentation associatedwith the mental state of the user. The computer-executable componentscan further comprise auxiliary information component that determinesauxiliary information associated with the content based on the mentalstate of the user for rendering at a user device in association with thepresentation of the media presentation.

In another embodiment, a device is provided that can comprise a memorythat stores computer-executable components and a processor that executescomputer-executable components stored in the memory. In one or moreimplementations, the computer-executable components can comprise afeedback component that receives feedback information for a user duringa media presentation and regarding a mental state of the user during themedia presentation. The computer-executable components can furthercomprise a communication component that receives auxiliary informationabout a portion of content of the media presentation associated withreception of the feedback information. The computer-executablecomponents can further comprise a rendering component that renders theauxiliary information on a display of the device during the mediapresentation and based on the portion of the content.

According to another embodiment, a computer-implemented method isprovided. In one example, the computer-implemented method comprisesdetermining, by a device operatively coupled to a processor, feedbackinformation for a user in response to a media presentation, wherein thefeedback information is indicative of a mental state of the user duringpresentation of the media presentation at a user device associated withthe user. The computer-implemented method can further comprisedetermining, by the device, content of the media presentation associatedwith the mental state of the user, and determining, by the device,auxiliary information about the content based on the mental state of theuser. The computer-implemented method can further comprise providing, bythe device, the auxiliary information to the user device during thepresentation of the media presentation and for rendering at the userdevice in association with presentation of the content.

DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of an example, non-limiting systemthat facilitates conditional provisioning of auxiliary information witha media presentation in accordance with one or more embodimentsdescribed herein.

FIG. 2 presents a table of an example, non-limiting auxiliaryinformation index in accordance with one or more embodiments describedherein.

FIG. 3 illustrates a block diagram of an example, non-limiting userdevice that facilitates conditional provisioning of auxiliaryinformation with a media presentation video in accordance with one ormore embodiments described herein.

FIG. 4 illustrates a flow diagram of an example, non-limiting processthat facilitates conditional provisioning of auxiliary information witha media presentation to a user based on a mental state of the user inaccordance with one or more embodiments described herein.

FIG. 5 illustrates a block diagram showing presentation of differentauxiliary information associated to different users based on differentintellectual abilities of the users in accordance with one or moreembodiments described herein.

FIG. 6 illustrates a block diagram of another example, non-limitingsystem that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein.

FIG. 7 illustrates a block diagram of another example, non-limitingsystem that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein.

FIG. 8 illustrates a block diagram of another example, non-limitingsystem that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein.

FIG. 9 illustrates a block diagram of another example, non-limitingsystem that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein.

FIG. 10 illustrates a diagram of an example, non-limiting heat mapcorresponding to different mental states of different users responsiveto a presentation in accordance with one or more embodiments describedherein.

FIG. 11 illustrates a flow diagram of an example, non-limitingcomputer-implemented method that facilitates conditional provisioning ofauxiliary information with a media presentation in accordance with oneor more embodiments described herein.

FIG. 12 illustrates a flow diagram of another example, non-limitingcomputer-implemented method that facilitates conditional provisioning ofauxiliary information with a media presentation in accordance with oneor more embodiments described herein.

FIG. 13 illustrates a block diagram of an example non-limiting operatingenvironment in which one or more embodiments described herein can befacilitated.

DETAILED DESCRIPTION

The following detailed description is merely illustrative and is notintended to limit embodiments and/or application or uses of embodiments.Furthermore, there is no intention to be bound by any expressed orimplied information presented in the preceding Background or Summarysections, or in the Detailed Description section.

The subject disclosure is directed to computer processing systems,computer-implemented methods, apparatus and/or computer program productsthat facilitate providing auxiliary information associated with apresentation to a user device for a user that is viewing, listening to,or otherwise receiving the presentation. The auxiliary information canbe based on feedback provided by the user during the presentationindicating a need or desire for the auxiliary information. Thepresentation can include for example, a live or recorded educational,informative, or instructional presentation in which one or morepresenters speak about a particular subject and/or provide a visualdemonstration related to the subject. For example, a presentation caninclude a live or recorded lecture presented by a professor to a groupof students about atomic bonding while presenting and discussing apreviously prepared slideshow on atomic bonding. In another example, apresentation can include a live or recorded audible description ofdifferent art included in a museum in association with a walking tourthrough the museum.

In particular, as a user is viewing, listening to or otherwise receivinga presentation, depending on the nature of the presentation, the usermay be interested in or intrigued by certain parts of the presentation,confused by certain parts of the presentation, bored with certain partsof the presentation, etc. When the presentation is experienced in a liveenvironment, especially in a group setting, the user may be unable ortoo shy to ask the presenter questions associated with parts of thepresentation the user finds interesting or confusing. Similarly, theuser may be unable or too shy to inform the presenter that the userconsiders certain subject matter boring, not useful or otherwise notengaging in contexts in which the user or the presenter could elect tochange the content or a part of the content of the presentation todifferent content the user would likely find more interesting, useful orengaging. Likewise, when experiencing a presentation in a non-livecontext, such as watching a video of a presentation, listening to anaudio recording of a presentation, or viewing a slideshow, the user maysee content the user finds interesting, confusing, boring, etc. However,the user may not be inclined or able to stop the presentation and lookup additional information (e.g., via searching sources on the Internet)or change the presentation (e.g., stop the presentation entirely, skipover content etc.).

In accordance with one or more embodiments, the subject computerprocessing systems, computer-implemented methods, apparatus and/orcomputer program products facilitate automatically (e.g., without directhuman involvement) providing auxiliary information to a user device fora user viewing, listening to, or otherwise experiencing a presentationand based on feedback received from the user during the presentationindicating a mental state of the user during the presentation. Inparticular, techniques are provided for automatically detecting a mentalstate of a user during a presentation and correlating the mental statewith a particular part of the presentation. For example, in someimplementations, the mental state of a user can be detected based onfacial expressions exhibited by the user during the presentation thatindicate the user is confused, interested, bored, etc. In oneembodiment, the facial expressions are captured and/or detected via aheads-up display (HUD) device worn by the user or another deviceassociated with the user. In some embodiments, a user can also provide,via the user device, an explicit request for auxiliary informationrelated to a part of a presentation the user is confused about,interested in, bored with, etc. while the user is experiencing thepresentation. In another embodiment, the user can provide explicit inputthat states the user is confused, interested, bored etc. via the userdevice in association with presentation of a particular part of apresentation.

In various implementations, a presentation system can receive thefeedback and access information previously associating known parts ofthe presentation with auxiliary information, such as clarifyinginformation intended to clarify user confusion about a particular partof the presentation or supplementary information intended to provideadditional detailed examples about a particular part of the presentationfor users that express interest in that particular part of thepresentation. The system can further retrieve the auxiliary informationand provide it to the user device of the user for displaying at the userdevice, playing at the user device, or otherwise rendering at the userdevice. In one embodiment, the auxiliary information can be provided tothe user device and presented at the user device in real-time such thatthe auxiliary information is presented during presentation of theparticular part of the presentation associated with the cause of theuser mental state. For example, in response to reception of feedbackindicating a user is confused about topic 2 of a presentation,clarifying information about topic 2 can automatically be provided tothe user device and rendered or displayed at the user device (e.g., aHUD device, a tablet, a smartphone, a laptop computer, a desktopcomputer etc.). In another embodiment, the auxiliary information can beprovided to the user after the presentation is completed or during abreak period in the presentation.

In some implementations, different versions of the auxiliary informationcan be available that are tailored to different user traits orpreferences, such as different learning styles, educational levels,intellectual levels or abilities, reading levels, etc. The differentversions can vary in the type of content of the auxiliary information,which can include, but is not limited to, text, images, video, audio andhyperlinks. The different versions can also vary with respect to anamount of content, a degree of complexity of the content, anintellectual level associated with the content, and the like. Accordingto these embodiments, in addition to selecting auxiliary informationrelated to a mental state of the user, a particular version of theauxiliary information can be selected based on a trait or preference ofthe user.

The various features and functionalities of one or more of theimplementations of the disclosed computer processing systems,computer-implemented methods, apparatus and/or computer program productscan allow users to experience a presentation in a live or non-livesetting and automatically receive auxiliary information when desired orneeded as determined based on implicit (e.g., a determined mental state)or explicit (e.g., a direct request for the auxiliary information)feedback obtained via the user device during the presentation.Accordingly, a plurality of users can view or experience the samepresentation in a live or non-live setting and receive differentauxiliary information at different times during the presentation basedon their individual responses to different parts of the presentationand/or their individual traits and preferences. Therefore, a samepresentation can be tailored to the needs and preferences of differentusers in real-time as the respective users experience the presentation,thus enabling each user to experience the presentation in a morefulfilling and enjoyable manner. Further, in a live setting, differentusers can be provided with clarifying information, supplementaryinformation, attention information, etc. without asking the presenterquestions that the user may be too shy to ask or that may not berelevant to other users experiencing the presentation. As a result, thepresenter can more efficiently and effectively provide the presentationto all users without stopping the presentation to answer questions.

One or more embodiments are now described with reference to thedrawings, wherein like referenced numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea more thorough understanding of the one or more embodiments. It isevident, however, in various cases, that the one or more embodiments canbe practiced without these specific details.

FIG. 1 illustrates a block diagram of an example, non-limiting system100 that facilitates conditional provisioning of auxiliary informationwith a media presentation in accordance with one or more embodimentsdescribed herein. Aspects of systems (e.g., system 100 and the like),apparatuses or processes explained in this disclosure can constitutemachine-executable component(s) embodied within machine(s), e.g.,embodied in one or more computer readable mediums (or media) associatedwith one or more machines. Such component(s), when executed by the oneor more machines, e.g., computer(s), computing device(s), virtualmachine(s), etc. can cause the machine(s) to perform the operationsdescribed.

The system 100 and/or the components of the system 100 can providesubstantial improvements to live and Internet based learning systems byautomatically providing users with auxiliary information related to apresentation in real-time or substantially real-time based on the users'mental and emotional states and/or preferences. Accordingly, a pluralityof users can view or experience the same presentation in a live ornon-live setting and receive different auxiliary information atdifferent times during the presentation based on their individualresponses to different parts of the presentation and/or their individualtraits and preferences. Therefore, a same presentation can bedynamically and automatically tailored to the needs and preferences ofdifferent users in real-time as the respective users experience thepresentation, thus enabling each user to experience the presentation ina more fulfilling and enjoyable manner. Further, in a live setting,auxiliary information can be presented to a user in a HUD as the user iswatching, listening to or otherwise experiencing the presentation,thereby minimizing distraction associated with consuming the auxiliaryinformation during the presentation.

The system 100 and/or the components of the system 100 can be employedto use hardware and/or software to solve problems that are highlytechnical in nature, that are not abstract and that cannot be performedas a set of mental acts by a human. For example, system 100 and/or thecomponents of the system 100 can be employed to use hardware and/orsoftware to perform operations including affective computing related toautomatically detecting and recognizing emotional information,correlating the emotional information with presentation contentassociated with causation of a specific mental state, and automaticallyselecting and providing auxiliary information for the presentationcontent. Further, some of the processes performed may be performed byspecialized computers for carrying out defined tasks related to theperforming affective computing to facilitate automatic provision ofauxiliary content to users based on emotional state. System 100 and/orcomponents of the system 100 can be employed to solve new problems thatarise through advancements in technology, computer networks, theInternet and the like. System 100 can further provide technicalimprovements to live and Internet based learning systems by improvingprocessing efficiency among processing components associated withselecting and providing auxiliary information associated with apresentation in real-time based a user's current mental state andpreferences.

As shown in FIG. 1, system 100 can include a presentation server device102, one or more networks 126 and one or more user devices 128. Thepresentation server device 102 can include various computer-executablecomponents, including, but not limited to, server feedback component104, analysis component 106, auxiliary information component 110 andserver communication component 116. The presentation server device 102can also include or otherwise be associated with at least one memory 120that stores computer-executable components (e.g., the server feedbackcomponent 104, the analysis component 106, the auxiliary informationcomponent 110, and the server communication component 116). Thepresentation server device 102 can also include or otherwise beassociated with at least one processor 118 that executes thecomputer-executable components stored in the memory 120. Thepresentation server device 102 can further include a system bus 114 thatcan couple the various components including, but not limited to, theserver feedback component 104, the analysis component 106, the auxiliaryinformation component 110, the server communication component 116,memory 120 and/or processor 118.

In various embodiments, the presentation server device 102 and the oneor more user devices 128 can operate in a server/client relationshipwherein auxiliary information associated with a presentation is providedby the presentation server device 102 to the one or more user devices128 based on processing, by the presentation server device 102 and/orthe one or more user devices 128, of feedback information regarding theneed or desire for the auxiliary information by respective usersassociated with the one or more user devices 128.

In some implementations, the presentation server device 102 and the oneor more user devices 128 can be communicatively coupled via one or morenetworks 126. Such networks 126 can include wired and wireless networks,including, but not limited to, a cellular network, a wide area network(WAN) (e.g., the Internet) or a local area network (LAN). For example,the one or more user devices 128 can communicate with presentationserver device 102 (and vice versa) using virtually any desired wired orwireless technology, including but not limited to: wireless fidelity(Wi-Fi), global system for mobile communications (GSM), universal mobiletelecommunications system (UMTS), worldwide interoperability formicrowave access (WiMAX), enhanced general packet radio service(enhanced GPRS), third generation partnership project (3GPP) long termevolution (LTE), third generation partnership project 2 (3GPP2) ultramobile broadband (UMB), high speed packet access (HSPA), Zigbee andother 802.XX wireless technologies and/or legacy telecommunicationtechnologies, BLUETOOTH®, Session Initiation Protocol (SIP), ZIGBEE®,RF4CE protocol, WirelessHART protocol, 6LoWPAN (IPv6 over Low powerWireless Area Networks), Z-Wave, and/or an ultra-wideband (UWB) standardprotocol.

For example, in one embodiment, one or more user devices 128 can berespectively associated with students located in relatively closeproximity to the presentation server device 102 (e.g., within the samephysical classroom space). According to this embodiment, the one or moreuser devices 128 and the presentation server device 102 can becommunicatively coupled via an area network (e.g., LAN). In anotherembodiment, the one or more user devices 128 can be respectivelyassociated with students located in disparate remote physical locationsrelative to the presentation server device 102. According to thisembodiment, the one or more user devices 128 and the presentation serverdevice 102 can be communicatively coupled via a WAN (e.g., a cellularnetwork, the Internet, etc.). In an aspect, one or more components ofsystem 100 can interact via disparate networks.

The presentation server device 102 can include server communicationcomponent 116 to facilitate wireless communication between thepresentation server device 102 and the one or more user devices 128, andbetween the presentation server device 102 and one or more otherexternal devices (not shown). For example, the server communicationcomponent 116 can receive feedback information from one or more userdevices 128 and/or one or more other devices (e.g., a remote camera,sensors worn by the user, etc.) and forward the feedback information tothe server feedback component 104 for processing. The servercommunication component 116 can also send auxiliary information to theone or more user devices 128 for rendering at the respective userdevices in association with viewing, listening to, or otherwiseexperiencing a presentation. In some embodiments, the servercommunication component 116 can also send presentation data to the oneor more user devices. For example, the server communication component116 can send slides included in a slideshow component of a presentationfor rendering at the respective user devices. In another example inwhich the presentation includes a video, the server communicationcomponent 116 can send the video to the respective user devices. In yetanother example in which the presentation includes an audio recording,the server communication component 116 can send the audio recording tothe respective devices. The server communication component 116 can be orinclude hardware (e.g., a central processing unit (CPU), a transceiver,a decoder), software (e.g., a set of threads, a set of processes,software in execution) or a combination of hardware and software thatfacilitates communicating information between the presentation serverdevice 102 and the one or more user devices 128.

Although in system 100 various components (e.g., the server feedbackcomponent 104, the analysis component 106, the auxiliary informationcomponent 110, and the server communication component 116) are providedon a server device (e.g., a presentation server device 102), in otherembodiments, any number of different types of devices can be associatedwith or include the aforementioned components. All such embodiments areenvisaged. Further, although in the embodiment shown the server feedbackcomponent 104, the analysis component 106, the auxiliary informationcomponent 110, and the server communication component 116 are providedat the presentation server device 102, it should be appreciated that thearchitecture of system 100 is not so limited. For example, one or moreof the components included at the presentation server device 102 can belocated at another device, such as another server device, anintermediary device between the server device and the one or more userdevices 128, or at the one or more user devices 128, etc.

The one or more user devices 128 can include any suitable computingdevice associated with a user and that can receive and/or renderauxiliary information associated with a presentation provided by thepresentation server device 102. In some implementations, the one or moreuser devices 128 can also facilitate capturing feedback informationregarding the respective user need or desire for the auxiliaryinformation and providing the feedback information to the presentationserver device 102 (e.g., as discussed infra with respect to FIG. 3). Forexample, the one or more user devices 128 can include a desktopcomputer, a laptop computer, a television, an Internet enabledtelevision, a mobile phone, a smartphone, a tablet user computer (PC), adigital assistant (PDA), a HUD, a virtual reality (VR) headset, anaugmented reality (AR) headset, or another type of wearable computingdevice. As used in this disclosure, the terms “user,” “student,”“presenter,” and the like refer to a person, entity, system, orcombination thereof that can employ system 100 (or additional systemsdescribed in this disclosure) using a user device of the one or moreuser devices 128 or the presentation server device 102.

The presentation server device 102 can facilitate conditionalprovisioning of auxiliary information with a media presentation. Theterm “media presentation” is used herein to describe a live or recorded(e.g., an audio recording, a video recording, or a video recordingincluding audio) presentation that includes a visual component, an audiocomponent, or both a visual and an audio component. In variousimplementations, the media presentation can be educational, informativeor instructive in nature. For example, a media presentation can includea live or recorded lecture presented by a teacher to group of students.In some implementations, the live or recorded lecture can be presentedin association with a slideshow having one or more slides, such as aslideshow presentation created using POWERPOINT® or a similar softwareapplication. The information included in the respective slides caninclude text, charts, images, video, audio, hyperlinks, or the like. Inanother example, a media presentation can include a live or recordeddemonstration performed by one or more people. For instance, a mediapresentation can include a live or recorded presentation of anexperiment (e.g., a laboratory experiment), a procedure (e.g., asurgical procedure), or an activity (e.g., cooking, snowboarding). Inanother example, a media presentation can include a live or recordedpresentation in which one or more presenters describe and/or demonstratehow to do an activity (e.g., a “how-to” video). In yet another example,a media presentation can include a live or recorded presentation inwhich a tour guide walks through a museum and provides informativeinformation about different art in the museum. Still in another example,a media presentation can include an audio recording of the tour throughthe museum. According to this example, the audio recording can belistened to by museum visitors using headphones associated via awearable device as they walk through the museum and auxiliaryinformation can be rendered via the wearable device based on feedbackreceived from the respective visitors in association with presentationof the audio recording.

In order to facilitate providing auxiliary information about a mediapresentation to a user device for a user viewing, listening to orotherwise watching the presentation at the user device (e.g., of the oneor more user devices 128), the server feedback component 104 can receivefeedback information for the user during the presentation regarding theuser need or desire for the auxiliary information. In one or moreembodiments, this feedback information can include information regardingone or more mental states of the user during review of the presentation.For example, the server feedback component 104 can receive feedbackinformation that indicates the user is intrigued or particularlyinterested, that the user is confused, that the user is bored orunengaged, that the user is amused, that the user is excited, that theuser is concerned or worried, that the user is offended, etc. Based onthe determined mental state of the user during a presentation, theauxiliary information component 110 can determine whether the user needsor desires auxiliary information for a particular part of thepresentation associated with the cause of the mental state. For example,based on a determination that the user is confused in association withpresentation of particular topic of a presentation, the auxiliaryinformation component 110 can determine clarifying auxiliary informationto provide to the user at the user device and that is intended to easethe user confusion on the topic. In some implementations, the servercommunication component 116 can further provide the auxiliaryinformation to the user device for rendering in association with thetopic of the presentation (e.g., while the topic is being presented orduring a break period following presentation of the topic).

In some implementations, feedback regarding a mental state of the usercan be based on one or more facial expressions expressed by the userduring the presentation. For example, system 100 can employ variousexisting emotion recognition technologies that can determine a usermental or emotional state based on analysis of facial expressions andeye movement of the user captured in video or one or more images of theuser. With facial emotion detection, algorithms can detect faces withina photo or video, and sense micro expressions by analyzing therelationship between points on the face, based on curated databasescompiled in academic environments. According to these implementations,as a user is viewing, listening to, or otherwise experiencing apresentation, facial expressions and/or eye movements of the user can becaptured and/or detected via one or more user facing cameras. Forexample, in some embodiments, the user can wear a HUD device including auser facing camera or image sensor that can capture facial expressionsand/or eye movements of the user during a presentation. In anotherexample, a user facing camera can be included on a device such as atablet, smartphone, desktop or laptop computer employed by the user andcapture facial expressions and/or eye movements of the user during apresentation. In another example, an external user facing camera can beincluded in a same room as the user or a group of users and capturefacial expressions and/or eye movements of the respective users. In someembodiments, the user facing camera or cameras can be or be provided atthe one or more user devices 128 associated with the respective users atwhich auxiliary information for the presentation is rendered.

In various embodiments, image data of a user face during a presentationcan be captured and processed to determine a mental state of the user inreal-time or substantially real-time. As used herein, the term“real-time” can mean processing and capturing to determine the mentalstate within a defined number of minutes or seconds (e.g., within 10seconds, within 30 seconds, within 60 seconds, within 2 minutes) afterthe image data is generated. Accordingly, as the user is viewing,listening to, or otherwise experiencing a presentation, one or moremental states of the user over the course of the presentation can bedetermined in real-time or substantially real-time. In some embodiments,captured facial expressions and/or eye movements can be processed by thecapture device (e.g., a user device of the one or more user devices 128or another device) that captured the respective facial expressionsand/or eye movements to determine a mental or emotional state of theuser. The capture device can further send feedback informationidentifying the determined mental or emotional state of the user to thepresentation server device 102 in real-time or substantially real-time.According to these embodiments, the server feedback component 104 canreceive information that identifies respective mental states of the userover the course of a presentation in real-time or substantiallyreal-time. For example, in one embodiment, a camera or image sensor canbe included at a user device associated with a user (e.g., a user deviceof the one or more user devices 128). The user device can capture (e.g.,via the camera or image sensor) and process facial expressions and/oreye movements of the user during the presentation to determine a mentalstate of the user (e.g., interested, confused, excited, bored, amused,offended, etc.). The user device can further provide informationidentifying the mental state of the user to the presentation serverdevice 102 in real-time or substantially real-time (in response tocapture and processing). In other embodiments, image data of a user facecaptured during a presentation (e.g., by a camera or image sensor at theuser device or another device) can be sent to the presentation serverdevice 102 by the capture device for processing by the server feedbackcomponent 104. For example, a user device (e.g., a HUD device, a tablet,a laptop computer, a desktop computer, etc.) associated with a userexperiencing a presentation can capture image data of the user faceduring the presentation and send the image data (e.g., in real-time orsubstantially real-time) to the presentation server device 102 forprocessing by the server feedback component 104. According to theseembodiments, the server feedback component 104 can receive image data(e.g., video or still images) captured of a user face during apresentation and process the image data to determine respective mentalor emotional states of the user during the presentation in real-time orsubstantially real-time.

In various additional embodiments, feedback information regarding amental or emotional state of a user during a presentation can bedetermined based on analysis of speech spoken by the user during thepresentation. According to these embodiments, the user device employedby the user, the presentation server device 102, or another devicewithin audible range of the user can include audio recording hardwareand software to record and analyze speech spoken by the user during apresentation. For example, the speech can include a question raised bythe user. In another example, the speech can include dialogue betweenthe user and the presenter or between the user and another user. In someimplementations, analysis of the speech can include analysis of tone andword content using one or more sonic algorithms to determine a mental oremotional state of the user during the presentation. Similar to theimage data, the server feedback component 104 can receive processedspeech data from the capture device associated with capture of themotion data (e.g., the user device of the one or more user devices 128,the presentation server device 102, or another device) identifying amental or emotional state of the user based on the speech data, and/orreceive raw speech data from the capture device for processing by theserver feedback component 104 to determine the mental or emotional stateassociated with the speech data.

In another embodiment, feedback information regarding a mental oremotional state of a user during a presentation can be determined basedon analysis of body movement or gestures of the user during thepresentation. For example, motion information regarding movement andmotion of the user during a presentation can be captured and analyzed toidentify gestures or body language indicative of different emotionalstates of the user during the presentation. According to thisembodiment, in one aspect, one or more motion sensors can be worn by theuser or be included in a device worn by the user (e.g., the user deviceof the one or more user devices 128 or another device) and capturemotion data regarding motion or movement of the user during thepresentation (e.g., gestures, fidgeting, remaining relatively still,changing a body position, blinking, foot tapping, etc.). In anotherexpect, information regarding user motion during presentation can bediscerned from image data (e.g., video) captured of the user during thepresentation. The server feedback component 104 can further receiveprocessed motion data from the capture device associated with capture ofthe motion data (e.g., a user device of the one or more user devices 128or another device) identifying a mental or emotional state of the userbased on the motion data, and/or receive raw motion data for processingby the server feedback component 104 to determine the mental oremotional state associated with the motion data.

In another embodiment, feedback information regarding a mental oremotional state of a user during a presentation can be determined basedon analysis of biometric data captured from the user during thepresentation. For example, the biometric information can include, but isnot limited to, information regarding the user heart rate, respiratoryrate, muscle tension, and hormone levels (e.g., cortisol, oxytocin,acetylcholine, dopamine, serotonin, gaba, glutamine, endorphin,epinephrine, norepinephrine, and glutamate). According to thisembodiment, one or more biometric sensors can be worn by the user (e.g.,external to the body and/or internal to the body), and/or be included ina device worn by the user (e.g., a user device of the one or more userdevices 128 or another device) and biometric data associated with theuser during the presentation. The server feedback component 104 canfurther receive processed biometric data from the capture deviceassociated with capture of the biometric data (e.g., the user device)identifying a mental or emotional state of the user based on thebiometric data, and/or receive raw biometric data for processing by theserver feedback component 104 to determine the mental or emotional stateassociated with the biometric data.

In addition to determining a user mental state based on feedbackinformation including one or more facial expressions of the user, speechof the user, motion of the user and/or biometric information of the userreceived in response to the presentation, the server feedback component104 can also receive explicit user feedback regarding a user request forauxiliary information associated with a presentation. For example, asthe user is viewing, listening to or otherwise experiencing apresentation, the user can provide direct input requesting auxiliaryinformation associated with a presentation that is received by theserver feedback component 104. For example, the user device (e.g., auser device of the one or more user devices 128 or another device) caninclude a suitable input mechanism (e.g., selection of a hard or softbutton, a touch screen, a keypad, a keyboard, a mouse, voice input,gesture input, etc.) via which the user can provide direct inputindicating a mental state of the user (e.g., input stating or meaning“I'm confused”) and/or a type of auxiliary information the user wouldlike to receive (e.g., input stating or meaning “I want clarifyinginformation,” “I want supplementary information,” etc.). For example,the user can select a button on the user device, provide verbal ortextual input, perform a gesture or generate another defined signal thatcan be received by the server feedback component 104 and indicate theuser is confused about a particular part of the presentation and wouldlike to receive clarifying information. Similarly, the user can select adifferent button on the user device, provide different verbal or textualinput, perform a different gesture, or generate a different definedsignal that can be received by the server feedback component 104 andindicate the user is interested in a particular part of the presentationand would like to receive clarifying information.

After the server feedback component 104 receives and/or determinesfeedback regarding a user mental state and/or explicit desire forauxiliary information during a presentation, in one or more embodiments,the analysis component 106 can determine the particular content of thepresentation for which the user has indicated desire or need forauxiliary information. In particular, the analysis component 106 caninclude content association component 108 to correlate feedbackindicating a particular mental state of the user (e.g., interested,confused, bored, excited, amuse, not amused, etc.) or desire forauxiliary information with a specific part of the content included inthe presentation that is being presented at the time the feedback isreceived. For instance, in one or more embodiments, the presentationserver device 102 can have access to information identifying contentincluded in a presentation (e.g., in memory 120 or at another device).For example, in embodiments in which the presentation is a livepresentation, the presentation server device 102 can have access toinformation that identifies different topics and/or sub-topics includedin the presentation and content respectively associated with thedifferent topics and/or sub-topics. In another example in which thepresentation includes a live or recorded presentation that includes aslideshow, the presentation server device 102 can have access toinformation identifying content included in respective slides of theslideshow and/or content associated with different parts or elements ofa single slide.

In some embodiments, the content association component 108 can determinea part of the content included in a presentation that is associated withreception of user feedback based on timing of reception of the feedbackand a current time point or time frame associated with the presentation.For example, the presentation can include a live or recordedpresentation associated with a known duration wherein particular partsor content included in the presentation are associated with known timepoints or time frames over the duration of the presentation. Forexample, with respect to a presentation including a plurality of knowntopics identified as topic 1, topic, 2, topic 3, etc., each (or, in someembodiments, one or more) of the different topics can be associated withknown time points or time frames throughout the presentation.Information regarding content respectively associated with differenttime points or time frames of the presentation can be stored in memory120 or otherwise accessible to the content association component 108.Accordingly, as a presentation is being presented, the contentassociation component 108 can determine a time point or time frame ofthe presentation associated with reception of the feedback. For example,in some implementations, the presentation server device 102 can receiveinformation identifying the start time of the presentation and track thepresentation time following the start of the presentation. In anotherexample, in implementations in which the presentation includes a video,the presentation server device 102 can receive information identifying acurrent time in the video when the feedback is received. In someembodiments, the presentation server device 102 can play the video orstream the video to the user's user device and thus correlate receivedfeedback with a current time point or time frame of the video. Thecontent association component 108 can further identify the particularcontent of the presentation (e.g., topic 1, topic, 2, topic, 3, etc.)associated with that time point or time frame.

In another embodiment in which the presentation includes a slideshow,the content association component 108 can have access to information(e.g., stored in memory 120 or at another device) identifying content ofthe presentation respectively associated with each slide (or, in someembodiments, one or more slides). The content association component 108can further determine or receive information identifying a current slidethat is being presented during a presentation at a time when thefeedback is received. For example, in some embodiments, the presentationserver device 102 can provide or render the respective slides and thushave direct knowledge about what slide is currently being presented. Insome implementations, the content association component 108 can furtherhave access to information identifying sub-topics or elements in a sameslide and determine the particular sub-topic or element of a same slidethat is being presented at the time feedback is received. For example,the slides can include interactive slides in which different elements orparts of a single slide can be activated or highlighted. According tothis implementation, the content association component 108 can determinethe particular sub-topic or element being presented at the time feedbackis received based on information indicating a particular part or elementof the slide is being pointed to, highlighted, selected or otherwiseactivated.

Still in other embodiments, the presentation server device 102 caninclude or have access to information associating known keywords in apresentation with specific parts of the content of the presentation.According to these embodiments, the server feedback component 104 canreceive or determine information identifying a known keyword that isspoken during the presentation at a time associated with reception ofthe feedback. For example, in embodiments in which the presentationserver device 102 is located within audible range of the presenter, thecontent association component 108 can include or employ speechrecognition hardware and software to capture speech spoken by thepresenter and identify keywords spoken throughout the presentation. Thecontent association component 108 can further determine a keyword (orkeywords) spoken at time associated with reception of the feedback andfurther correlate the feedback with the known part of the contentassociated with the keyword (or keywords). In some embodiments, the userdevice associated with the user (e.g., a user device of the one or moreuser devices 128 or another device) can include the speech recognitionhardware and/or software and determine the keyword or keywords beingpresented at a time when the user generates or provides the feedbackregarding the user mental state or explicit need or desire for auxiliaryinformation. According to these embodiments, the user device can includeinformation identifying the keyword or keywords in association withproviding the feedback information to the server feedback component 104.

Once the analysis component 106 has determined a particular part of thecontent included in a presentation associated with received feedback,the auxiliary information component 110 can facilitate selectingauxiliary information for provision to the user at the user device basedon the feedback. In particular, in one or more embodiments, thepresentation server device 102 can include or have access to definedauxiliary information respectively associated with known parts of thepresentation (e.g., topics, subtopic, elements, etc.). For example, thepresentation server device 102 can include an auxiliary informationdatabase 124 stored in memory 120 (or at another device accessible tothe presentation server device 102) including a variety of definedauxiliary information for a presentation. The auxiliary information caninclude, but is not limited to, text, images, charts, audio, video,and/or hyperlinks that include auxiliary information related torespective parts of the presentation.

In one or more embodiments, various types of auxiliary information canbe associated with defined mental states. For example, in someimplementations, auxiliary information intended to facilitate helping auser better understand a part of the presentation the user is confusedabout is referred to herein as “clarifying” auxiliary information. Inanother example, auxiliary information intended to provide a user moredetailed information or examples about a part of the presentation a useris particularly interested in or intrigued by is referred to herein as“supplementary” auxiliary information. In another example, auxiliaryinformation intended to capture a user's attention (e.g., when the userprovides feedback indicating the user is bored or uninterested) isreferred to herein as “attention” auxiliary information. It should beappreciated that the above described auxiliary information types aremerely exemplary and other types of auxiliary information related todifferent mental states, purposes, and/or content types are alsoenvisioned.

In addition to having different types of auxiliary information fordifferent mental states, in some embodiments, different auxiliaryinformation can also be associated with defined user traits andpreferences. For example, in one or more embodiments, the presentationserver device 102 can include or have access to user profile informationfor respective users to which a presentation is presented. The userprofile information can include information that identifies or describesa type of auxiliary information the user prefers. For example, in someimplementations, the user profile information can indicate whether theuser prefers visual auxiliary information or textual auxiliaryinformation. In another example, the user profile information canindicate whether the user prefers complex auxiliary information, simpleauxiliary information, short auxiliary information, long auxiliaryinformation, etc. With regard to educational presentations, the userprofile information can include, but is not limited to, informationregarding a user's intellectual level or capability, grades, learningtype (e.g., visual, mathematical, kinesthetic, auditory, etc.), readinglevel or speed of the user, degree of background knowledge in thepresentation subject matter, multitasking ability, and the like. Thepresentation server device 102 can further determine and tailor traitsand preferences for a user over time using machine learning techniques.

Thus in various embodiments, in addition to having different types ofauxiliary information for different mental states, different versions ofauxiliary information of a same type (e.g., clarifying, supplementary,attention, etc.) can be provided for a same part of content included ina presentation, wherein the different versions are associated withdefined user traits and/or preferences. The different versions forexample can include different types of content (e.g., text, images orvideo), different amounts of content, different degrees of contentcomplexity, and the like. For example, a first version of clarifyingauxiliary information regarding how two molecules bind can include avisual animation including three-dimensional models of the two moleculesdemonstrating how they bind, and second version of clarifying auxiliaryinformation regarding how the two molecules bind can include chemicalformulas of the two molecules when separated and when combined.According to this example, the first version of the clarifyinginformation may be provided for a visual learner with a basic chemicalbackground while the second version may be provided for mathematicallearner with a more complex chemical background. In another example, afirst version of supplementary auxiliary content that is intended forpresentation to a slow reader can include less text information than asecond version of the supplementary auxiliary content that is intendedfor presentation to a fast reader.

In the embodiment shown, the presentation server device 102 can alsoinclude an auxiliary information index 122 stored in the memory 120 (orotherwise accessible to the presentation server device 102). In variousimplementations, the auxiliary information index 122 can includeinformation that identifies a particular presentation and known contentparts (e.g., topics, sub-topics, elements, etc.) of the presentation. Inembodiments in which the presentation includes slides, the auxiliaryinformation index 122 can include information identifying respectiveslides associated with each content part (or, in some embodiments, withone or more content parts). In embodiments in which the presentationincludes a video, the auxiliary information index can includeinformation identifying time points or time frames of the videoassociated with each content part (or, in some embodiments, with one ormore content parts). In other embodiment in which the presentationincludes speech, the auxiliary information index can include keywordsrespectively associated with different parts of the presentation. Theauxiliary information index 122 can also include information identifyingauxiliary information (e.g., included in auxiliary information database124) respectively associated with each part of the presentation, a typeof the auxiliary information, a version of the auxiliary information,and/or a user profile trait or preference associated with the auxiliaryinformation.

For example, FIG. 2 presents a table of an example, non-limitingauxiliary information index 200 in accordance with one or moreembodiments described herein. Repetitive description of like embodimentsemployed in respective embodiments is omitted for sake of brevity.

The auxiliary information index 200 is provided for a presentationtitled “Ionic Bonding.” In one implementation, the presentation wasprepared by a teacher for presenting to a group of students. Thepresentation includes 10 defined topics associated with seven preparedslides. The auxiliary information index 200 can include informationidentifying each topic and the respective slides with which each topicis associated. The auxiliary information index 200 can further includekeywords associated with each topic (or, in some embodiments, with oneor more topics). In some implementations, the keywords can be detectedduring the presentation in speech spoken during the presentation (e.g.,by the presenter) using voice recognition hardware and software (e.g.,provided at the user device or the presentation server device 102) tofacilitate identifying the current topic being presented. In addition,the auxiliary information index 200 can include information identifyingdifferent auxiliary information respectively associated with each topic.In the embodiment shown, each topic has two versions of clarifyinginformation and two versions of supplementary information associatedtherewith. In this example embodiment, the first versions of theclarifying and supplementary auxiliary information can be intended forprovision to students that are visual learners and the second versionsof the clarifying and supplementary auxiliary information can beintended for provision to students that are mathematical learners. Eachauxiliary information entry is associated with a unique identifier(e.g., C-101, C-103, C-103, S-101, S-102, S-103) to facilitateidentifying and retrieving the corresponding auxiliary information inthe auxiliary information database 124. For example, in response to adetermination that a visual learning student is confused in associationwith presentation of topic 7, the auxiliary information component 110can retrieve the auxiliary information associated with identifier C-114.

With reference back to FIG. 1, the auxiliary information component 110can include selection component 112 to facilitate selecting auxiliaryinformation to provide to a user at the user's user device based onreceived feedback for the user during a presentation. In one or moreembodiments, based on information received and/or determined by theserver feedback component 104 identifying a mental state of the user orproviding an explicit request for a type of auxiliary information,information determined by content association component 108 identifyinga specific part of the presentation associated with a cause of themental state of the user, and/or information regarding a known trait(e.g., intellectual level, learning type, etc.) and/or preference of theuser, the selection component 112 can employ the auxiliary informationindex 122 to identify, determine or select the appropriate auxiliaryinformation (e.g., included auxiliary information database 124) forproviding to the user. For example, based on reception of user feedbackinformation indicating the user has a confused mental state inassociation with presentation of topic 2 of a presentation, theselection component 112 can access and employ the auxiliary informationindex 122 to determine whether clarifying auxiliary information has beenprovided for topic 2 in the auxiliary information database 124. Inresponse to a determination that clarifying information has beenprovided for topic 2, the selection component 112 can retrieve theauxiliary information and the auxiliary information component 110 canprovide (e.g., using server communication component 116) the auxiliaryinformation to the user device (e.g., (e.g., a user device of the one ormore user devices 128 or another device) associated with the user forrendering at the user device. Further, in an embodiment in whichdifferent versions of the auxiliary information associated withdifferent user traits or preferences are provided, the selectioncomponent 112 can determine one or more traits and/or preferences of theuser and select the version of the auxiliary information that is bestsuited for the user based on the user's traits and/or preferences. Forexample, if the user is a visual learner, the selection component 112can select the version of the auxiliary information for topic 2 that isassociated with visual learning. For instance, the visual learningversion may include pictorial examples in lieu of another version of theauxiliary information for topic 2 that includes mathematical examples.

In one or more embodiments, the auxiliary information component 110 canprovide (e.g., using server communication component 116) selectedauxiliary information associated with a part of a presentation to a userdevice (e.g., a user device of the one or more user devices 128 oranother device) of a user as or after the feedback is received and thepart of the presentation is being presented. The user device can furtherrender the auxiliary information in association with presentation of thepart of the presentation (e.g., as the part is discussed in a livecontext, as the part is presented in a live-context, as the part ispresented or played in a slideshow or video in a live or non-liveenvironment, etc.). For example, if the auxiliary information isassociated with a particular topic being presented, the user device canrender the auxiliary information while the topic is being presented (or,in some embodiments, within a defined amount of time after the topic ispresented). In other embodiments, the auxiliary information can berendered at a later time, such as during a break period after the topicis discussed or after the entire presentation is finished.

The manner in which the auxiliary information is rendered at the userdevice (e.g., a user device of the one or more user devices 128) canvary depending on the features and functionalities of the user device, atype of content of the auxiliary information (e.g., text, images, video,audio, a hyperlink, etc.), a timing of rendering, and/or whether avisual component (e.g., a slideshow, a video, an animation, an image,etc.) associated with the presentation is also being rendered at theuser device. For example, if the auxiliary information includes audio(e.g., a spoken description, a sound, a song, etc.) associated with avisual part of the presentation, the auxiliary information can berendered via a speaker or earpiece of the user device as the userwatches or experiences a visual part of the presentation. In anotherexample, if the auxiliary information includes text, images, video, or ahyperlink, the auxiliary information can be displayed via a displayscreen of the user device. In one example embodiment, the user devicecan include a HUD device worn by a user as the user watches and/orexperiences a presentation and the auxiliary information can be renderedin the HUD during presentation of the part of the presentation for whichthe auxiliary information is provided. For example, the user device caninclude a HUD device that is worn by a user while located within aclassroom and viewing a slideshow presentation presented by a professor.The slideshow presentation can be displayed on a primary display screenat the front of the classroom. Accordingly, as the user is viewing theprimary display screen at the front of the classroom, auxiliaryinformation can be presented in the HUD and the user can continue tolook at the primary display while also viewing the auxiliaryinformation. Thus the auxiliary information will essentially appear asan overly onto the primary display at the front of the classroom.Further, in an embodiment in which the presentation includes slides orvideo that is also displayed via the display screen of the user device,the auxiliary information can be rendered in a pop-up display window orbillboard as an overlay on the slide or video content or in an area ofthe display outside a display window including the slide or videocontent. In another example, the auxiliary information can be renderedin a different window or tab.

FIG. 3 illustrates a block diagram of an example, non-limiting userdevice 300 that facilitates conditional provisioning of auxiliaryinformation with a media presentation video in accordance with one ormore embodiments described herein. In various embodiments, user device300 can be or include a user device of the one or more user devices 128presented in system 100. Repetitive description of like embodimentsemployed in respective embodiments is omitted for sake of brevity.

User device 300 can include an input device 302, client feedbackcomponent 312, client communication component 314, rendering component316 and a display 320. The user device 300 can also include or otherwisebe associated with at least one memory 324 that storescomputer-executable components (e.g., the client feedback component 312,the client communication component 314, and the rendering component316). The user device 300 can also include or otherwise be associatedwith at least one processor 322 that executes the computer-executablecomponents stored in the memory 120. The user device 300 can furtherinclude a device bus 318 that can couple the various componentsincluding, but not limited to, the input device 302, the client feedbackcomponent 312, the client communication component 314, the display 320,the memory 324 and/or processor 322.

In one or more embodiments, the client feedback component 312 canreceive and/or process user feedback information regarding an interestor desire for auxiliary content associated with a presentation. Forexample, the client feedback component 312 can receive feedbackinformation regarding a mental state of a user associated with userdevice 300 during a presentation. In some implementations, such userfeedback can be captured by the user device 300 via an input device 302provided at the user device. For example, the input device 302 caninclude but is not limited to, a camera 304, an audio capture device306, one or more motion sensors 308 and/or one or more biometricsensors. In one implementation, the camera 304 can be a user facingcamera that captures imagery (e.g., video and/or still images) of theuser face during a presentation. In some embodiments, the camera 304 canalso capture video and/or images of the user that can be processed todetect and identify motion of the user (e.g., blinking of eyes, noddingof head, etc.). The audio capture device 306 can include a microphone oranother type of audio capture device that can receive and record audioduring a presentation, such as speech spoken by the user associated withuser device 300, speech of a presenter of the presentation, dialoguebetween the user and the presenter, and/or dialogue between the user andanother user. In some implementations, the audio capture device 306 canfurther process captured audio to convert detected speech to text. Theone or more motion sensors 308 can include, for example, anaccelerometer and/or a gyroscope that can detect motion of the userdevice 300 when worn, held or otherwise operated by the user. The one ormore biometric sensors 310 can include biometric sensors that detectbiometric information for the user during a presentation, including, butnot limited to, heart rate, respiratory rate, and hormone levels of theuser. In some implementations, one or more of the motion sensors 308and/or the biometric sensors can be external to the user device 300(e.g., worn by the user, implanted within the user, etc.) andcommunicatively coupled to the user device 300.

In some embodiments, feedback information captured via the input device302 can be received and processed by the client feedback component 312to determine a mental state of the user based on the feedback. Forexample, the client feedback component 312 can determine a mental stateof the user based on analysis of one or more facial expressions includedin imaged data captured via the camera, based on analysis of tone ofvoice and words spoken in speech of the user captured via audio capturedevice 306, based on analysis of motion data regarding body language ofthe user captured via the one or more motion sensors 308, and/or basedon analysis of biometric data for the user captured via the one or morebiometric sensors 310. According to these embodiments, the clientcommunication component 314 can send the processed feedback dataidentifying a mental state of the user to the presentation server device102 to facilitate receiving auxiliary information from the presentationserver device 102. In some embodiments, the sending can be performed inreal-time or substantially real-time. In some embodiments, the clientcommunication component 314 can send the presentation server device 102raw feedback information captured via the input device 302 forprocessing by the server feedback component 104. The clientcommunication component 314 can include same or similar features andfunctionalities as server communication component 116. For example, inone or more embodiments, the client communication component 314 caninclude hardware and/or software that facilitate wired or wirelesscommunication between the user device 300 and the presentation serverdevice 102, and between the user device 300 and one or more otherexternal devices (not shown).

In some embodiments, the client feedback component 312 can also receiveexplicit input (e.g., via the input device 302) indicating a mentalstate of the user and/or a desire for a type of auxiliary informationassociated with a presentation. For example, using an input mechanismprovided by the input device 302, the user can generate a defined signalthat indicates the user is confused, interested, bored, etc. at anygiven time during a presentation. Similarly the user can provide inputthat includes an explicit request for a type of auxiliary information(e.g., “please provide me with clarifying information on this topic”).It should be appreciated that the input mechanism can vary. For example,the user can provide speech input, motion gesture input (e.g., blinkingof eye, nodding of head, tapping of foot, raising of hand, etc.), aswell as input via a hard or soft button, a keypad, a touch screen, amouse, etc. The client feedback component 312 can also receive suchexplicit input and the client communication component 314 can providethis explicit input to the presentation server device 102.

The client communication component 314 can further receive auxiliaryinformation from the presentation server device 102 based on thefeedback information provided to the presentation server device 102. Insome implementations, the client communication component 314 can alsoreceive the presentation or a part of the presentation. For example, theclient communication component 314 can download the presentation or apart of the presentation for rendering at the user device 300 and/orreceive the presentation or a part of the presentation as streamed mediafor rendering at the user device 300.

The rendering component 316 can render auxiliary information receivedfrom the presentation server device 102. For example, in an embodimentin which the auxiliary information includes audio, the renderingcomponent 316 can cause the audio to be played at the user device 300via a speaker of the user device (e.g., not shown). In another example,in an embodiment in which the auxiliary information includes visual data(e.g., text, images, video, hyperlinks, etc.) the rendering component316 can generate a graphical user interface that can be displayed viathe display 320 of the user device 300 and including received auxiliaryinformation. The appearance of the auxiliary information can varydepending on the features and functionalities of the user device 300 andthe type of content of the auxiliary information. In one embodiment, thedisplay 320 includes a HUD device and the auxiliary information can berendered in the HUD device. In some embodiments, the rendering component316 can also render the presentation at the user device 300 (e.g., viadisplay 320 and/or via a speaker of the user device). In embodiment inwhich the presentation includes visual content (e.g., a slideshow, avideo, etc.) that is rendered via the display 320, in someimplementations, the rendering component 316 can render the auxiliaryinformation as an overly on the visual content (e.g., via a pop-updisplay window or a billboard).

FIG. 4 illustrates a flow diagram of an example, non-limiting process400 that facilitates conditional provisioning of auxiliary informationwith a media presentation to a user based on a mental state of the userin accordance with one or more embodiments described herein. Repetitivedescription of like embodiments employed in respective embodiments isomitted for sake of brevity.

In the embodiment shown, a user is wearing glasses that function as aHUD device while experiencing a media presentation. For example, theuser may be in classroom setting attending a live lecture presented by aprofessor and that may include a slideshow presentation. In anotherexample, the user may be walking about a museum while listening to anaudio recording presentation regarding artwork in the museum. In anotherexample, the user may be at home and experiencing the media presentationvia HUD device. The presentation, for example, may include aneducational video or a recorded lecture.

At 401, the presentation begins. In the implementation shown, thepresentation includes four topics for which information identifying thetopics and auxiliary information for at least topic 2 have beenestablished prior to beginning the presentation (e.g., and provided tothe presentation server device 102). In the implementation shown, eachof the 4 topics is associated with a different slides or a single slideincluded in a slideshow element of the media presentation. In anotherexample, each (or, in some embodiments, one or more) of the 4 topics maybe identified in an outline of the presentation provided to thepresentation server device 102 in association with auxiliary informationfor the respective topics. At 402, as the presentation is beingpresented, it is detected that the user has a confused mental statebased on one or more of the types of feedback discussed herein (e.g.,via client feedback component 312 and/or server feedback component 104.For instance in one example, the HUD device can capture visual imageryof the user face and the client feedback component 312 or the serverfeedback component 104 can determine a mental state of confusion of theuser based on one or more facial expression of the user included in theimagery. At 404, the topic associated with the user's mental state ofconfusion is determined using one or more of the mechanisms describedherein (e.g., via the content association component 108). Then at 405,clarification text is provided to the user at the user's HUD device(e.g., via auxiliary information component 110 and server communicationcomponent 116) and rendered via the HUD device.

FIG. 5 illustrates a block diagram showing presentation of differentauxiliary information associated to different users for a same topic(e.g., topic 2) and based on different intellectual abilities of theusers in accordance with one or more embodiments described herein.Repetitive description of like embodiments employed in respectiveembodiments is omitted for sake of brevity.

For example, in the embodiment shown, two different users, amathematical learner and a visual learner, provided feedback informationin association with viewing, listening to, or otherwise experiencing apresentation indicating they both were confused about topic 2 of thepresentation. Based on the feedback information, auxiliary information501 including a mathematical explanation of topic 2 can be presented tothe mathematical learner. On the other hand, auxiliary information 502including s a visual clarifying explanation of topic 2 can be presentedto the visual learner.

FIG. 6 illustrates a block diagram of another example, non-limitingsystem 600 that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein. System 600 includes same or similarfeatures as system 100 with the addition of timing component 602.Repetitive description of like elements employed in respectiveembodiment is omitted for sake of brevity.

In various embodiments, the auxiliary information component 110 canprovide auxiliary information to a user device (e.g., a user device ofthe one or more user devices 128) for rendering in association withpresentation of the particular part of presentation that the auxiliaryinformation is associated with. For example, if the auxiliaryinformation is associated with topic 2 of a presentation, the auxiliaryinformation component 110 can provide (e.g., using server communicationcomponent 116) the auxiliary information to the user device for rendingin real-time in association with presentation of topic 2. For example,if the auxiliary information includes visual information (text, images,video, a hyperlink, etc.) it can be displayed via a display screen ofthe user device. In some implementations in which a visual component ofthe presentation is also displayed at the user device (e.g., aslideshow, a video, etc.) the auxiliary information can be rendered inan overlay window or billboard onto the visual component of thepresentation.

In association with rendering auxiliary information during apresentation and particularly in association with a specific part of thepresentation, the timing component 602 can determine timing forpresentation of the auxiliary information (e.g., a start time andduration for presentation of the auxiliary information). In one or moreembodiments, the timing for presentation of auxiliary information can bebased on the amount of content included in the auxiliary information,wherein larger amounts of content are presented for longer periods oftime. In another implementation, the timing for presentation ofauxiliary information can be cued such that the auxiliary informationfor a particular topic is rendered as soon as it is received at the userdevice (e.g., in response to receiving user feedback by the serverfeedback component 104) and removed or dismissed from being displayedupon the beginning of a new topic, sub-topic, subject, slide, etc. Inanother implementation, the timing for presentation of auxiliaryinformation can be based on a character trait of preference of the user.For example, the timing component 602 can determine duration forpresentation of auxiliary information based on a reading level of theuser, an intellectual capability or level of the user, or multi-taskingability of the user. For instance in some embodiments, users associatedwith a relatively low reading level or intellectual capability (e.g.,with respect to threshold level) can be provided with auxiliaryinformation that is displayed for a longer duration relative users witha relatively high reading level or intellectual capability, such thatthe users with the lower reading level or intellectual capability areprovided more time to review the auxiliary information. Still in otherembodiments, the duration of display of auxiliary information can becontrolled by the user viewing the auxiliary information. For example,the user can select and control when to minimize or dismiss display ofthe auxiliary information.

In various additional embodiments, auxiliary information that isprovided to a user device in association with a presentation can becollected and retained (e.g., in memory 324 at the user device and/oraccessed at the presentation server device 102) for later review by theuser after the presentation is complete or during break periods betweenthe beginning and end of the presentation. In some additionalembodiments, the timing component 602 can determine that certainauxiliary information is not suitable for display during thepresentation but during a planned break period. For example, dependingon the length and complexity of the auxiliary information, a type ofcontent of the auxiliary information (e.g., a video, or hyperlink to ascientific article, etc.), a trait or preference of the user, and amulti-tasking ability of the user, the timing component 602 candetermine whether to display the auxiliary information during thepresentation or during a planned break period. For example, the timingcomponent 602 can determine that auxiliary information requiringsignificant time for review or attention of the user should be saved forpresentation during a break period in which the user has more time toreview the auxiliary information with the full attention of the user. Insome implementation, auxiliary information rendered during a breakperiod can be displayed in a larger window (e.g., using the fulldimensions of the user device display) relative to a window thatincludes auxiliary information when rendered during the presentation.

In one implementation, in response to a determination that the auxiliaryinformation should be displayed during a planned break period, theauxiliary information component 110 can notify the user during thepresentation that auxiliary information for a particular topic,sub-topic element, etc., has been selected for the review by the userduring the upcoming break period. For example, the user can receive anaudible signal or visual signal that is rendered at the user device inassociation with the particular topic, sub-topic, element etc.Accordingly, the user will know ahead of time that he or she will getclarifying information or supplementary information about the particulartopic, sub-topic, etc. soon and to relax and enjoy the presentation.When the break period arrives, the auxiliary information component 110can then send the auxiliary information to the user device forpresentation, or the user device can receive the auxiliary informationat an earlier time and wait to display the auxiliary information untilthe break period based on timing metadata associated with the auxiliaryinformation by the timing component 602 identifying the time when theauxiliary information should be displayed.

FIG. 7 illustrates a block diagram of another example, non-limitingsystem 700 that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein. System 700 includes same or similarfeatures as system 600 with the addition of streaming component 702.Repetitive description of like elements employed in respectiveembodiment is omitted for sake of brevity.

In some embodiments, the presentation server device 102 can employstreaming component 702 to stream auxiliary information to the one ormore user devices 128 based on received feedback for users associatedwith the respective user devices indicating a need or desire for theauxiliary information. For example, rather than providing the auxiliaryinformation to user devices as a download, the auxiliary information canbe retained at the presentation server device 102 and streamed to a userdevice in association with a presentation via the streaming component702. In some implementations, the auxiliary information can include avideo, an animation, or audio that can be streamed by the presentationserver device 102 to a user device when called for. In addition, in oneor more implementations, the presentation server device 102 can alsostream a presentation to the one or more user devices. For instance, thestreaming component can stream a visual and/or audio component of amedia presentation to the one or more user devices for concurrentpresentation of the visual and/or audio component of the mediapresentation at the respective devices. For example, in animplementation in which the presentation includes a video, the streamingcomponent 702 can stream the video to the one or more user device forconcurrent playback at the respective user devices. In another example,in which the presentation includes a slideshow that is presented inassociation with a live or recorded description of the slides by apresenter, the streaming component 702 can stream the slideshow to theone or more user device for concurrent playback at the respective userdevices in association with reception of speech associated provided bythe presenter. Still in other embodiments, the streaming component 702can stream live presentations filmed at remote locations relative to theone or more user devices for concurrent playback at the one or more userdevices 128.

FIG. 8 illustrates a block diagram of another example, non-limitingsystem 800 that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein. System 800 includes same or similarfeatures as system 700 with the addition of inference component 802.Repetitive description of like elements employed in respectiveembodiment is omitted for sake of brevity.

Inference component 802 can aid in various inferences or determinationsassociated with aspects of presentation server device 102. For example,inference component 802 can facilitate the server feedback component 104and/or the client feedback component 312 in association with inferring amental state of a user based on feedback regarding facial expression ofthe user, speech of the user, motion or body movement of the user, andbiometric data of the user captured during a presentation. The inferencecomponent 802 can further aid in determining what part of a presentationa user's feedback is directed to. The inference component 802 canfurther facilitate the auxiliary information component 110 inassociation with inferring or determining what auxiliary information toprovide to a particular user and when (e.g., timing information) toprovide the auxiliary information.

In order to provide for or aid in the numerous inferences describedherein, inference component 802 can examine the entirety or a subset ofthe data to which it is granted access and can provide for reasoningabout or infer states of the system (e.g., system 800 and the like),environment, etc. from a set of observations as captured via eventsand/or data. An inference can be employed to identify a specific contextor action, or can generate a probability distribution over states, forexample. The inference can be probabilistic (e.g., the computation of aprobability distribution over states of interest can be based on aconsideration of data and events). An inference can also refer totechniques employed for composing higher-level events from a set ofevents and/or data.

Such an inference can result in the construction of new events oractions from a set of observed events and/or stored event data, whetheror not the events are correlated in close temporal proximity, andwhether the events and data come from one or several event and datasources. Various classification (explicitly and/or implicitly trained)schemes and/or systems (e.g., support vector machines, neural networks,expert systems, Bayesian belief networks, fuzzy logic, data fusionengines, etc.) can be employed in connection with performing automaticand/or inferred action in connection with the claimed subject matter.

A classifier can map an input attribute vector, x=(x1, x2, x4, x4, xn),to a confidence that the input belongs to a class, such as byf(x)=confidence(class). Such classification can employ a probabilisticand/or statistical-based analysis (e.g., factoring into the analysisutilities and costs) to prognose or infer an action that a user desiresto be automatically performed. A support vector machine (SVM) is anexample of a classifier that can be employed. The SVM operates byfinding a hyper-surface in the space of possible inputs, where thehyper-surface attempts to split the triggering criteria from thenon-triggering events. Intuitively, this makes the classificationcorrect for testing data that is near, but not identical to trainingdata. Other directed and undirected model classification approachesinclude, e.g., naïve Bayes, Bayesian networks, decision trees, neuralnetworks, fuzzy logic models, and probabilistic classification modelsproviding different patterns of independence can be employed.Classification as used herein also is inclusive of statisticalregression that is utilized to develop models of priority.

FIG. 9 illustrates a block diagram of another example, non-limitingsystem 900 that facilitates conditional provisioning of auxiliaryinformation with a media presentation in accordance with one or moreembodiments described herein. System 900 includes same or similarfeatures as system 800 with the addition of suggestion component 902 tothe auxiliary information component 110 and a display 904. Repetitivedescription of like elements employed in respective embodiment isomitted for sake of brevity.

In some embodiments, a presenter of a presentation or an assistant ofthe presenter (or another suitable entity) can play an active role inproviding auxiliary information to users at their respective userdevices 128 during a presentation. For example, in many of thepreviously described embodiments, the auxiliary information component110 can automatically select (e.g., via selection component 112)auxiliary information and provide the auxiliary information to a userbased on reception, via server feedback component 104, of feedbackinformation from the user indicating a need or desire for the auxiliaryinformation in association with a particular part of a presentation. Invarious alternative embodiments, rather than automatically providing theauxiliary information to a user device of the user, the suggestioncomponent 902 can provide the presenter information identifying the userand indicating the mental state of the user or identifying an explicitrequest received from the user for a type of auxiliary information. Insome implementations, the suggestion component 902 can further provideinformation identifying the auxiliary information that is associatedwith the part of the presentation associated with the received feedback,and the traits/preferences of the user (e.g., the auxiliary informationselected by the selection component 112 based on the above notedfactors). The suggestion component 902 can further suggest the presenterprovide the user with the identified auxiliary information.

For example, in one or more implementations, the presentation serverdevice 102 can be accessed and operated by the presenter either directlyor indirectly (e.g., using another device to interface with thepresentation server device 102). In an implementation in which thepresenter can access the presentation server device directly, thepresentation server device 102 can include a display 904 via whichinformation provided by the suggestion component 902 can be displayed tothe presenter. In an embodiment in which the presenter interfaces withthe presentation server device 102 via another device (e.g., a usercomputer, a tablet, a classroom computer, etc.), the informationprovided by the suggestion component 902 can be rendered via a displayat the other device. For example, in an embodiment in which thepresenter is presenting a presentation to a group of users, thesuggestion component 902 can provide the presenter with a display thatis regularly updated throughout the presentation indicating the mentalstates of the respective users.

FIG. 10 illustrates a diagram of an example, non-limiting heat map 1000corresponding to different mental states of different users responsiveto a presentation in accordance with one or more embodiments describedherein. Repetitive description of like elements employed in otherembodiments described herein is omitted for sake of brevity.

In particular, the heat map 1000 depicts the mental states of thedifferent users (e.g., Amanda, Bob, Catherine . . . Rachael) in responseto topic 2 of a presentation. Those users associated with a light greycoloration are considered to have a neutral mental state, thoseassociated with a white coloration are considered to have an interestedmental state, and those associated with a dark gray coloration areconsidered to have a confused mental state. In an exemplary embodiment,the heat map 1000 can be generated by the presentation server device 102based on information received from the server feedback component 104regarding mental states of the different users during the presentation.The heat map 1000 can further be rendered to a presenter of thepresentation (or another suitable entity) via a display (e.g., display904 or a display associated with another device that can interface withthe presentation server device 102).

With reference back to FIG. 9, based on reception of information fromthe suggestion component 902 regarding a mental state of a user, a topicof the presentation associated with the mental state, and/or auxiliaryinformation suitable and available for providing to the user, thepresenter can choose whether and when to provide the auxiliaryinformation to the user. For example, the presenter can provide inputthat can be received by the auxiliary information component 110requesting the auxiliary information be sent to the user or provideinput requesting the auxiliary information not be sent to the user.Based on input indicating the presenter wants the auxiliary informationsent to the user, the auxiliary information component 110 can thenproceed to provide the auxiliary information to the user as requested.In some embodiments, the presenter can also choose what auxiliaryinformation to provide to the user or create auxiliary information toprovide to the user. Further, in situations in which a majority of agroup of user (e.g., most of the students in the class), have a similarmental reaction to a particular part of the presentation, the presentercan choose to change the content of the presentation or render theauxiliary information to every user in the group. The presenter canfurther discuss the auxiliary information in more detail in a livesetting rather than providing the auxiliary information to the users forthe users to parse through on their own.

FIG. 11 illustrates a flow diagram of an example, non-limitingcomputer-implemented method 1100 that facilitates conditionalprovisioning of auxiliary information with a media presentation inaccordance with one or more embodiments described herein. Repetitivedescription of like elements employed in other embodiments describedherein is omitted for sake of brevity.

At 1102, a device operatively coupled to a processor (e.g., presentationserver device 102) can determine feedback information for a user inresponse to a media presentation, wherein the feedback information isindicative of a mental state of the user during presentation of themedia presentation at a user device associated with the user. At 1104,the device can determine content of the media presentation associatedwith the mental state of the user (e.g., via content associationcomponent 108). At 1106, the device can determine auxiliary informationabout the content based on the mental state of the user (e.g., viaauxiliary information component 110). At 1108, the device can providethe auxiliary information to the user device during the presentation ofthe media presentation and for rendering at the user device inassociation with presentation of the content (e.g., via servercommunication component 116 of FIG. 1 and/or streaming component 702 ofFIG. 7).

FIG. 12 illustrates a flow diagram of another example, non-limitingcomputer-implemented method 1200 that facilitates conditionalprovisioning of auxiliary information with a media presentation inaccordance with one or more embodiments described herein. Repetitivedescription of like elements employed in other embodiments describedherein is omitted for sake of brevity.

At 1202, a device operatively coupled to a processor (e.g., user device300) can receive feedback information for a user during a mediapresentation and regarding a mental state of the user during the mediapresentation (e.g., via client feedback component 312 of user device300). At 1204, the device can receive auxiliary information about aportion of content of the media presentation associated with receptionof the feedback information (e.g., via client communication component314 of user device 300). At 1206, the device can render the auxiliaryinformation on a display of the device during the media presentation andbased on the portion of the content (e.g., via rendering component 316of user device 300).

In order to provide a context for the various aspects of the disclosedsubject matter, FIG. 13 as well as the following discussion are intendedto provide a general description of a suitable environment in which thevarious aspects of the disclosed subject matter can be implemented. FIG.13 illustrates a block diagram of an example, non-limiting operatingenvironment in which one or more embodiments described herein can befacilitated. Repetitive description of like elements employed in otherembodiments described herein is omitted for sake of brevity.

With reference to FIG. 13, a suitable operating environment 1301 forimplementing various aspects of this disclosure can also include acomputer 1312. The computer 1312 can also include a processing unit1314, a system memory 1316, and a system bus 1318. The system bus 1318couples system components including, but not limited to, the systemmemory 1316 to the processing unit 1314. The processing unit 1314 can beany of various available processors. Dual microprocessors and othermultiprocessor architectures also can be employed as the processing unit1314. The system bus 1318 can be any of several types of busstructure(s) including the memory bus or memory controller, a peripheralbus or external bus, and/or a local bus using any variety of availablebus architectures including, but not limited to, Industrial StandardArchitecture (ISA), Micro-Channel Architecture (MSA), Extended ISA(EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus(USB), Advanced Graphics Port (AGP), Firewire (IEEE 1394), and SmallComputer Systems Interface (SCSI). The system memory 1316 can alsoinclude volatile memory 1320 and nonvolatile memory 1322. The basicinput/output system (BIOS), containing the basic routines to transferinformation between elements within the computer 1312, such as duringstart-up, is stored in nonvolatile memory 1322. By way of illustration,and not limitation, nonvolatile memory 1322 can include read only memory(ROM), programmable ROM (PROM), electrically programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), flash memory, ornonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM).Volatile memory 1320 can also include random access memory (RAM), whichacts as external cache memory. By way of illustration and notlimitation, RAM is available in many forms such as static RAM (SRAM),dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM(DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), directRambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambusdynamic RAM.

Computer 1312 can also include removable/non-removable,volatile/nonvolatile computer storage media. FIG. 13 illustrates, forexample, a disk storage 1324. Disk storage 1324 can also include, but isnot limited to, devices like a magnetic disk drive, floppy disk drive,tape drive, Jaz drive, Zip drive, LS-100 drive, flash memory card, ormemory stick. The disk storage 1324 also can include storage mediaseparately or in combination with other storage media including, but notlimited to, an optical disk drive such as a compact disk ROM device(CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive (CD-RWDrive) or a digital versatile disk ROM drive (DVD-ROM). To facilitateconnection of the disk storage 1324 to the system bus 1318, a removableor non-removable interface is typically used, such as interface 1326.FIG. 13 also depicts software that acts as an intermediary between usersand the basic computer resources described in the suitable operatingenvironment 1301. Such software can also include, for example, anoperating system 1328. Operating system 1328, which can be stored ondisk storage 1324, acts to control and allocate resources of thecomputer 1312. System applications 1330 take advantage of the managementof resources by operating system 1328 through program modules 1332 andprogram data 1334, e.g., stored either in system memory 1316 or on diskstorage 1324. It is to be appreciated that this disclosure can beimplemented with various operating systems or combinations of operatingsystems. A user enters commands or information into the computer 1312through input device(s) 1336. Input devices 1336 include, but are notlimited to, a pointing device such as a mouse, trackball, stylus, touchpad, keyboard, microphone, joystick, game pad, satellite dish, scanner,TV tuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 1314through the system bus 1318 via interface port(s) 1338. Interfaceport(s) 1338 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 1340 usesome of the same type of ports as input device(s) 1336. Thus, forexample, a USB port can be used to provide input to computer 1312, andto output information from computer 1312 to an output device 1340.Output adapter 1342 is provided to illustrate that there are some outputdevices 1340 like monitors, speakers, and printers, among other outputdevices 1340, which require special adapters. The output adapters 1342include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 1340and the system bus 1318. It should be noted that other devices and/orsystems of devices provide both input and output capabilities such asremote computer(s) 1344.

Computer 1312 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1344. The remote computer(s) 1344 can be a computer, a server, a router,a network PC, a workstation, a microprocessor based appliance, a peerdevice or other common network node and the like, and typically can alsoinclude many or all of the elements described relative to computer 1312.For purposes of brevity, only a memory storage device 1346 isillustrated with remote computer(s) 1344. Remote computer(s) 1344 islogically connected to computer 1312 through a network interface 1348and then physically connected via communication connection 1350. Networkinterface 1348 encompasses wire and/or wireless communication networkssuch as local-area networks (LAN), wide-area networks (WAN), cellularnetworks, etc. LAN technologies include Fiber Distributed Data Interface(FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ringand the like. WAN technologies include, but are not limited to,point-to-point links, circuit switching networks like IntegratedServices Digital Networks (ISDN) and variations thereon, packetswitching networks, and Digital Subscriber Lines (DSL). Communicationconnection(s) 1350 refers to the hardware/software employed to connectthe network interface 1348 to the system bus 1318. While communicationconnection 1350 is shown for illustrative clarity inside computer 1312,it can also be external to computer 1312. The hardware/software forconnection to the network interface 1348 can also include, for exemplarypurposes only, internal and external technologies such as, modemsincluding regular telephone grade modems, cable modems and DSL modems,ISDN adapters, and Ethernet cards.

Embodiments of the present invention may be a system, a method, anapparatus and/or a computer program product at any possible technicaldetail level of integration. The computer program product can include acomputer readable storage medium (or media) having computer readableprogram instructions thereon for causing a processor to carry outaspects of the present invention. The computer readable storage mediumcan be a tangible device that can retain and store instructions for useby an instruction execution device. The computer readable storage mediumcan be, for example, but is not limited to, an electronic storagedevice, a magnetic storage device, an optical storage device, anelectromagnetic storage device, a semiconductor storage device, or anysuitable combination of the foregoing. A non-exhaustive list of morespecific examples of the computer readable storage medium can alsoinclude the following: a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), a static randomaccess memory (SRAM), a portable compact disc read-only memory (CD-ROM),a digital versatile disk (DVD), a memory stick, a floppy disk, amechanically encoded device such as punch-cards or raised structures ina groove having instructions recorded thereon, and any suitablecombination of the foregoing. A computer readable storage medium, asused herein, is not to be construed as being transitory signals per se,such as radio waves or other freely propagating electromagnetic waves,electromagnetic waves propagating through a waveguide or othertransmission media (e.g., light pulses passing through a fiber-opticcable), or electrical signals transmitted through a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network can comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device. Computer readable programinstructions for carrying out operations of various aspects of thepresent invention can be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions can executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer can be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection can be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) can execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to customize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions. These computer readable programinstructions can be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks. These computer readable program instructions can also be storedin a computer readable storage medium that can direct a computer, aprogrammable data processing apparatus, and/or other devices to functionin a particular manner, such that the computer readable storage mediumhaving instructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram block or blocks. Thecomputer readable program instructions can also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational acts to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams can represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks can occur out of theorder noted in the Figures. For example, two blocks shown in successioncan, in fact, be executed substantially concurrently, or the blocks cansometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While the subject matter has been described above in the general contextof computer-executable instructions of a computer program product thatruns on a computer and/or computers, those skilled in the art willrecognize that this disclosure also can or can be implemented incombination with other program modules. Generally, program modulesinclude routines, programs, components, data structures, etc. thatperform particular tasks and/or implement particular abstract datatypes. Moreover, those skilled in the art will appreciate that theinventive computer-implemented methods can be practiced with othercomputer system configurations, including single-processor ormultiprocessor computer systems, mini-computing devices, mainframecomputers, as well as computers, hand-held computing devices (e.g., PDA,phone), microprocessor-based or programmable consumer or industrialelectronics, and the like. The illustrated aspects can also be practicedin distributed computing environments where tasks are performed byremote processing devices that are linked through a communicationsnetwork. However, some, if not all aspects of this disclosure can bepracticed on stand-alone computers. In a distributed computingenvironment, program modules can be located in both local and remotememory storage devices.

As used in this application, the terms “component,” “system,”“platform,” “interface,” and the like, can refer to and/or can include acomputer-related entity or an entity related to an operational machinewith one or more specific functionalities. The entities disclosed hereincan be either hardware, a combination of hardware and software,software, or software in execution. For example, a component can be, butis not limited to being, a process running on a processor, a processor,an object, an executable, a thread of execution, a program, and/or acomputer. By way of illustration, both an application running on aserver and the server can be a component. One or more components canreside within a process and/or thread of execution and a component canbe localized on one computer and/or distributed between two or morecomputers. In another example, respective components can execute fromvarious computer readable media having various data structures storedthereon. The components can communicate via local and/or remoteprocesses such as in accordance with a signal having one or more datapackets (e.g., data from one component interacting with anothercomponent in a local system, distributed system, and/or across a networksuch as the Internet with other systems via the signal). As anotherexample, a component can be an apparatus with specific functionalityprovided by mechanical parts operated by electric or electroniccircuitry, which is operated by a software or firmware applicationexecuted by a processor. In such a case, the processor can be internalor external to the apparatus and can execute at least a part of thesoftware or firmware application. As yet another example, a componentcan be an apparatus that provides specific functionality throughelectronic components without mechanical parts, wherein the electroniccomponents can include a processor or other means to execute software orfirmware that confers at least in part the functionality of theelectronic components. In an aspect, a component can emulate anelectronic component via a virtual machine, e.g., within a cloudcomputing system.

In addition, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom context, “X employs A or B” is intended to mean any of the naturalinclusive permutations. That is, if X employs A; X employs B; or Xemploys both A and B, then “X employs A or B” is satisfied under any ofthe foregoing instances. Moreover, articles “a” and “an” as used in thesubject specification and annexed drawings should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form. As used herein, the terms “example”and/or “exemplary” are utilized to mean serving as an example, instance,or illustration. For the avoidance of doubt, the subject matterdisclosed herein is not limited by such examples. In addition, anyaspect or design described herein as an “example” and/or “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs, nor is it meant to preclude equivalent exemplarystructures and techniques known to those of ordinary skill in the art.

As it is employed in the subject specification, the term “processor” canrefer to substantially any computing processing unit or devicecomprising, but not limited to, single-core processors;single-processors with software multithread execution capability;multi-core processors; multi-core processors with software multithreadexecution capability; multi-core processors with hardware multithreadtechnology; parallel platforms; and parallel platforms with distributedshared memory. Additionally, a processor can refer to an integratedcircuit, an application specific integrated circuit (ASIC), a digitalsignal processor (DSP), a field programmable gate array (FPGA), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. Further, processors can exploit nano-scalearchitectures such as, but not limited to, molecular and quantum-dotbased transistors, switches and gates, in order to optimize space usageor enhance performance of user equipment. A processor can also beimplemented as a combination of computing processing units. In thisdisclosure, terms such as “store,” “storage,” “data store,” datastorage,” “database,” and substantially any other information storagecomponent relevant to operation and functionality of a component areutilized to refer to “memory components,” entities embodied in a“memory,” or components comprising a memory. It is to be appreciatedthat memory and/or memory components described herein can be eithervolatile memory or nonvolatile memory, or can include both volatile andnonvolatile memory. By way of illustration, and not limitation,nonvolatile memory can include read only memory (ROM), programmable ROM(PROM), electrically programmable ROM (EPROM), electrically erasable ROM(EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g.,ferroelectric RAM (FeRAM). Volatile memory can include RAM, which canact as external cache memory, for example. By way of illustration andnot limitation, RAM is available in many forms such as synchronous RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rateSDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM),direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), andRambus dynamic RAM (RDRAM). Additionally, the disclosed memorycomponents of systems or computer-implemented methods herein areintended to include, without being limited to including, these and anyother suitable types of memory.

What has been described above include mere examples of systems andcomputer-implemented methods. It is, of course, not possible to describeevery conceivable combination of components or computer-implementedmethods for purposes of describing this disclosure, but one of ordinaryskill in the art can recognize that many further combinations andpermutations of this disclosure are possible. Furthermore, to the extentthat the terms “includes,” “has,” “possesses,” and the like are used inthe detailed description, claims, appendices and drawings such terms areintended to be inclusive in a manner similar to the term “comprising” as“comprising” is interpreted when employed as a transitional word in aclaim. The descriptions of the various embodiments have been presentedfor purposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments. The terminologyused herein was chosen to best explain the principles of theembodiments, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

What is claimed is:
 1. A system, comprising: a memory that storescomputer-executable components; a processor that executes thecomputer-executable components stored in the memory, wherein thecomputer-executable components comprise: a feedback component thatmonitors a mental state of a user during provision of a presentation tothe user, wherein the presentation comprises a spoken description of atopic; an analysis component that determines, based on a determinationthat the mental state comprises a defined type of mental state includedin a set of two or more defined types of mental states, a portion of thetopic associated with causation of the defined type of mental state,wherein the determining the portion of the topic comprises: determiningone or more key terms included in the spoken description of the topicthat were spoken at a time associated with detection of the defined typeof mental state of the user; and determining the portion of the topicbased on the one or more key terms; an auxiliary information componentthat determines auxiliary information about the portion of the topicbased on the defined type of mental state; and a communication componentthat provides the auxiliary information to a device employed by the userduring the presentation for rendering by the device in association withpresentation of the portion of the topic.
 2. The system of claim 1,wherein the defined type of mental state comprises a state of confusionand wherein the auxiliary information component determines the auxiliaryinformation to comprise of clarifying information about the portion ofthe topic based on the defined type of mental state being the state ofconfusion.
 3. The system of claim 1, wherein the defined type of mentalstate comprises a state of intrigue and wherein the auxiliaryinformation component determines the auxiliary information to compriseof supplementary information about the portion of the topic based on thedefined type of mental state being the state of intrigue.
 4. The systemof claim 1, wherein the analysis component determines the mental stateof the user based on feedback information identifying one or more facialexpressions of the user made during the provision of the presentation tothe user.
 5. The system of claim 1, wherein the auxiliary informationcomponent further determines a type of the auxiliary information basedon a learning style of user.
 6. The system of claim 1, wherein theauxiliary information component further determines a type of theauxiliary information based on an intellectual level of the user.
 7. Thesystem of claim 1, wherein the auxiliary information comprises a textualdescription associated with the portion of the topic, and wherein theauxiliary information component further determines a length of thetextual description based on profile information associated with theuser.
 8. The system of claim 1, wherein the auxiliary informationcomprises a textual description associated with the portion of thetopic, and wherein the auxiliary information component furtherdetermines a length of the textual description based on an intellectuallevel of the user.
 9. The system of claim 1, wherein thecomputer-executable components further comprise: a streaming componentthat streams a visual component of the presentation to the device forconcurrent presentation of the visual component of the presentation atthe device in association with reception of the spoken description ofthe topic.
 10. The system of claim 1, wherein the device comprises ahead-mounted display that is worn by the user.
 11. Acomputer-implemented method, comprising: monitoring, by a deviceoperatively coupled to a processor, a mental state of a user duringprovision of a presentation to the user, wherein the presentationcomprises a spoken description of a topic; based on a determination thatthe mental state comprises a defined type of mental state included in aset of two or more defined types of mental states, determining, by thedevice, a portion of the topic associated with causation of the definedtype of mental state, wherein the determining the portion of the topiccomprises: determining one or more key terms included in the spokendescription of the topic that were spoken at a time associated withdetection of the defined type of mental state of the user; anddetermining the portion of the topic based on the one or more key terms;determining, by the device, auxiliary information about the portionbased on the defined type of mental state; and providing, by the device,the auxiliary information to a user device employed by the user duringthe presentation for rendering by the device in association withpresentation of the portion.
 12. The computer-implemented method ofclaim 11, wherein the defined type of mental state comprises a state ofconfusion and wherein the determining the auxiliary informationcomprises determining clarifying information about the portion.
 13. Thecomputer-implemented method of claim 11, wherein the defined type ofmental state comprises a state of intrigue and wherein the determiningthe auxiliary information comprises determining supplementaryinformation about the portion.
 14. The computer-implemented method ofclaim 11, wherein the monitoring the mental state of the user comprises:receiving feedback information identifying one or more facialexpressions of the user made during the provision of the presentation tothe user.
 15. The computer-implemented method of claim of claim 14,wherein the monitoring the mental state of the user further comprises:determining the mental state of the user based on the one or more facialexpressions.
 16. The computer-implemented method of claim 11, furthercomprising: determining a type of the auxiliary information based on alearning style of user.
 17. The computer-implemented method of claim 11,further comprising: determining a type of the auxiliary informationbased on an intellectual level of the user.
 18. A non-transitorycomputer program product for conditional provisioning of auxiliaryinformation with a presentation, the computer program product comprisinga computer readable storage medium having program instructions embodiedtherewith, the program instructions executable by a processor to causethe processor to: monitor, by the processor, monitoring, a mental stateof a user during provision of the presentation to the user, wherein thepresentation comprises a spoken description of a topic; determine, bythe processor, based on a determination that the mental state comprisesa defined type of mental state included in a set of two or more definedtypes of mental states, a portion of the topic associated with causationof the defined type of mental state, wherein the determining the portionof the topic comprises: determining one or more key terms included inthe spoken description of the topic that were spoken at a timeassociated with detection of the defined type of mental state of theuser; and determining the portion of the topic based on the one or morekey terms; determine, by the processor, auxiliary information about theportion based on the defined type of mental state; and provide, by theprocessor, the auxiliary information to a device employed by the userduring the presentation for rendering by the device in association withpresentation of the portion.
 19. The non-transitory computer programproduct of claim 18, wherein the defined type of mental state comprisesa state of confusion and wherein the processor determines the auxiliaryinformation to comprise clarifying information about the portion basedon the mental state being the state of confusion.